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Question 1 - Please tell us how we can get in touch with you if we need to 
discuss your comments further 


Name — Alexander Whalen 
Organisation — DIGITALEUROPE 


Email Address — alexander.whalen@digitaleurope.org 


Question 2 - Please tell us how you heard about this consultation 


Name and Date of event attended where consultation was invited — DIGITALEUROPE is a member of the expert 
Working Group and heard of the consultation directly from the Working Group facilitators 


Please reference email your received inviting you to comment — N/A 


Other — N/A 


Question 3 - Section 1: Introduction; Background and Regulatory Landscape 
| agree with the text of the draft — Yes and No 
Please make any comments on or suggested corrections to the text below or at the end if you need more space 


We welcome the inclusion of these sections as a way to set the framework so that readers understand why we 
are undertaking this exercise. This is the first step in clarifying the scope of the guidelines. However, we questions 
why the issue of ‘risk’ and ‘safety’ is brought up on page 1 as this does not accurately reflect the outcomes of the 
stakeholder meetings nor the reason for the creation of the guidelines. The scope of the exercise from the 
beginning was to address the quality of data for linking apps to EHRs. This exercise should therefore not cover 
‘safety’. Furthermore, while an overview/roadmap may be useful for readers during the drafting phase, we do 
not believe it is necessary for the final guidelines and as such should be removed prior to final publication. 


When it comes to the overview of the regulatory landscape, we believe that this section is extremely important 
so that readers (and Working Group Members) understand that we do not need to ‘re-invent the wheel’ on many 
issues. It also shows that any expansion of scope risks contradicting clear regulatory frameworks that already exist 
or are set to be implemented in the near future. However, we wish to express caution on the 3rd paragraph on 
page 10 (Section 1.2.4.1 Interfacing with medical devices legislation). Here the drafters begin to build up the “grey 
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zone”. The role of this guidance is NOT to define where the borders lie for a so-called “grey zone” as that would 
have implications on safety. If the Working Group decides that safety is of importance, then it should be for the 
European Commission and stated actors to define a process to effectively draw the line and address the points. 
Safety is the competence of the European Commission experts. It is not for an ad-hoc group (despite its best 
intentions) to declare what is safe and what is not. Moreover, the European Commission recently launched a 
consultation looking at the safety of non-embedded software. The consultation covers the Product Safety 
Directive, which is the right place to have a discussion on safety. 


Lastly, we wish to note that those standards mentioned on page 12 (Section 1.2.5 Existing standards) are 
principally those that relate directly to medical devices, which fall outside of the scope of the guidelines. This 
should be specified more clearly. Furthermore, to give a complete overview of standards, it might be worthwhile 
to includes those related to app security and privacy. Standards in this field exist and a reference could be useful 
to readers. Examples of standards to be mentioned are I1SO27034 for application security and ISO29151 & 
1SO29134 for privacy. 


Question 4 - Section 2: Purpose 
| agree with the text of the draft — Yes and No 
Please make any comments on or suggested corrections to the text below or at the end if you need more space 


As above, we believe that including a section on ‘purpose’ is important. We particularly welcome the first 
paragraph of this section as this reflects the main purpose of the guidelines. However, we object to the inclusion 
of paragraph 5 which states “better use of better apps for better healthcare”. While improving adoption is a 
laudable goal, that is not the objective of this exercise. We need to make sure that these guidelines do not seek 
to advocate for the use of certain apps over others. The potential increased use of mobile apps should only be 
treated as an indirect positive effect of the application of the guidelines, but not a direct objective. 


We also have concerns surrounding the language in paragraph 6. This is where the scope begins to extend and 
the mentioning of additional criteria emerges. Once again, while this is laudable, this is NOT the purpose of these 
guidelines and falls outside of the raison d’etre for why the group was established. This language must be 
narrowed as there is a risk that these ‘additional criteria’ would compete with other standards, other legislation, 
other projects, etc. This will create confusion and uncertainty. 
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Question 5 - Scope of the guidelines 
Feedback on the first draft reveals 2 different perspectives on scope: 
1. The scope should be limited to considering only validity and reliability. 


2. It is essential to consider a number of related criteria (9 are identified in this version of 
the draft guidelines). 


What do you think? 


| agree with a more limited scope (only validity and reliability). Please give reasons for your view — Yes. This is the 
reason for the establishment of the Working Group. It reflects the concerns raised in the responses to the Green 
Paper on mHealth Consultation and represents the core mandate behind the Commission’s call for expression 
when creating the group. Any expansion of this would require a new and extended working mandate from the 
Commission. 


| agree that it is also essential to consider a broader range of related criteria (as in the second draft) — No. We 
disagree STRONGLY with this. Not enough information/proof has been provided as to why we need to proceed 
with a broader range of criteria. The additional 9 criteria should to all apps, which begs the question as to why 
we are looking at them only in the context of mHealth apps. 


Please amplify or provide any other comments — As noted above, the additional criteria should in theory apply to 
all apps, which begs the question as to why they are being looked at in the context of lifestyle and wellbeing apps 
only? There is competition between apps and app stores to ensure that these criteria are met. If they are not 
met, then the apps will not be used. Market activities have also seen changes to app store policies showing that 
they respect these criteria. To make this exercise useful, we need to look at things that are of real relevance to 
mHealth apps. This requires a narrowing down of scope and focus to the issues which are particular to mHealth 
apps only. Expanding the criteria outside of the core problem we are trying to address dilutes this exercise. 


It is important to remember that DIGITALEUROPE, while only 1 member of the Working Group, represents 62 
companies and 37 national trade associations. Our members, both large and small, note that many of these 
criteria contradict clear and existing criteria set by OS manufactures and recognised by Member State 
Governments (and market practices). This will lead to confusion and uncertainty and has the risk of impacting the 
business operations of companies. 
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Question 6 - Section 4 of the guideline text identifies the main target groups 
expected to benefit from the guidelines: Citizens, mHealth Developers, App 
aggregators, Healthcare professionals and the healthcare system. It lists their 
perspectives on the shortcomings of the current situation and also their needs 
from the guidelines. 


Please comment by target group 


Current shortcomings and needs of citizens are correctly described. If not please comment — We do not believe 
citizens should be the target of this exercise. It is hard to believe that citizens will consult these guidelines prior 
to downloading or using and app. If the group thinks citizens should be a target, then a separate document (e.g. 
easy to reach brochure) needs to be developed to present in an easy way the criteria for choosing a good mHealth 
app. However, this needs to be the job or another Expert Group. 


Current shortcomings and needs of mHealth developers are correctly described. If not please comment — We do 
believe that the mHealth app developers are the right target for the guidelines. However, the incredibly long list 
of criteria which contradicts many of the criteria set out in OS manufactures guidelines plus the various legislative 
frameworks represents the existing shortcomings in this draft. 


Current shortcomings and needs of app aggregators are correctly described. If not please comment — We do not 
believe that app aggregators should be the target of this exercise. 


Current shortcomings and needs of healthcare professionals are correctly described. If not please comment — We 
do not believe that healthcare professionals should be the target of this exercise. While we recognise the difficulty 
faced by healthcare professionals when asked by patients if the should/should not use an app, it is unreasonable 
to expect healthcare professionals to go through this criteria to evaluate an app. The solving of this problem 
should be done by national health authorities to aggregate a list of recommended apps that meet the criteria and 
then have this list be shared with/accessible to healthcare professionals. 


Current shortcomings and needs of the healthcare system are correctly described. If not please comment — We do 
not believe that the healthcare system should be the target of this exercise. 


Please make any other comments on targeting - are we missing any important stakeholders? Are some important 
needs from the guidelines being overlooked? Is it possible for the guidelines to meet all the needs of the different 
stakeholders? — Along with mHealth app developers, we believe that public authorities should be one of the 
targets for these guidelines. We wish to stress that it is NOT possible for these guidelines to meet the needs of 
the different stakeholders. To target all of the above target groupings would require various iterations and 
versions of the guidelines. While we understand that many members of the Working Group have a desire to make 
sure that the guidelines address their target group, it is simply not feasible. If you narrow down the scope and 
purpose you narrow down the target groups and the exercise becomes easier. 
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Question 7 - Section 5: Format and Adoption 
| agree with the text of the draft — Yes and No 


Please make any comments on or suggested corrections to the text below or at the end if you need more space 
We agree that the guidelines should be simply to read, but if all the target groups are trying to be met, it will be 
impossible to create a ‘simple to read’ document. As stated above, if you narrow down the scope and purpose 
you can narrow down the target group and then the format/adoption exercise becomes easier. Right now we do 
not believe the different communication strategies for various audiences are feasible. We also wish to note that 
it would be good to emphasise the ‘voluntary’ nature of these guidelines within this section (and in the 
introduction). 


When it comes to a decision tree, we believe this only works for a narrow target audience. We would like to point 
back to the simplicity of the mHealth Privacy Code of Conduct as an example. The aim was an easy guide for app 
developers on how to make apps more privacy friendly. This included a clear and succinct check-list for them. If 
we had tried to take that document and addressed other target groups you would have a messy and unclear 
document. 


When it comes to the certification/labelling, we believe that if certification or labelling were to be brought up to 
a European level (or have mutual recognition in Member States), it could be an area to be explored. However, 
the requirement cannot fall onto app aggregators. They have current 50,000 apps coming into their stores per 
week. They do not have the expertise nor the resources to take part in certification. This is once again why we 
believe national authorities must be one of the target groups of this exercise. 


Question 8 - On the format of the guidelines 

The European Commission's Working Group is considering 
¢Practical support or guidance. 

eTerminology (clarity on use of terms). 

e Legal (clarity). 

Organisational or procedural approach. 

eCriteria (to be used in assessing app). 


Using the same target groups mentioned at Q6 above, please state which aspect will be 
most important. 


For citizens, the most important aspect is (choose from Practical support or guidance, Terminology, Legal, 
Organisational or procedural approach, Criteria. Explain why and how this could best be delivered. — We do not 
believe citizens should be the target of this exercise. It is hard to believe that citizens will consult these guidelines 
prior to downloading or using and app. 
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For mHealth developers, the most important aspect is (choose from Practical support or guidance, Terminology, 
Legal, Organisational or procedural approach, Criteria). Explain why and how this could best be delivered.— For 
mHealth app developers the focus should be on practical support and guidance. We do not believe the clarity, 
terminology, and particularly organisational (out of scope!) aspects are important. 


For app aggregators, the most important aspect is (choose from Practical support or guidance, Terminology, Legal, 
Organisational or procedural approach, Criteria). Explain why and how this could best be delivered. — We do not 
believe that app aggregators should be the target of this exercise. 


For healthcare professionals the most important aspect is (choose from Practical support or guidance, Terminology, 
Legal, Organisational or procedural approach, Criteria). Explain why and how this could best be delivered. — We do 
not believe that healthcare professionals should be the target of this exercise. 


For the health system, the most important aspect is (choose from Practical support or guidance, Terminology, Legal, 
Organisational or procedural approach, Criteria). Explain why and how this could best be delivered. — We do not 
believe that the healthcare system should be the target of this exercise. 


Please make any other comment on the form and purpose the guidelines here or at the end. — Once again we wish 
to note that the guidance should be targeted at public authorities. As stated above, the entire question depends 
on who the target group is. However, in our view, we believe it should focus on practical support and guidance. 


Question 9 - Section 6: Guidelines 
| agree with the text of the draft — No 


Please make any comments on or suggested corrections to the text below or at the end if you need more space We 
strongly disagree with this section. As previously noted we do not agree with the inclusion of the 9 criteria. They 
are not specific to mHealth apps and many of they have nothing to do with assessing the validity, reliability, and 
integrity of data that is produced by lifestyle and wellbeing apps. The scope in this section needs to be drastically 
reduced. We refer you to our previous consultation response where we outline why we do not think certain 
criteria are necessary in this exercise. 


We also continue to have serious reservations regarding Section 6.3 — Risk assessment. As noted in question 3, 
this group does not have the legitimacy and mandate to draw up where the borders lie for a so-called “grey zone”. 
Defining risk should remain the role of the European Commission and the established procedures. It not for this 
ad-hoc group to draw up risk definitions. We wish to draw an example with the current discussions around the 
GDPR. This Regulation is built on the foundation of a ‘risk-based approach’. However, the definition of risk is being 
left to the experts within the European Data Protection Authorities (together with industry expert stakeholders). 
It is not being left to an ad-hoc group. As such, we call for the deletion of this section of the guidelines. 
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Question 10 - Feed-back on the first draft of the guidelines indicates that we 
need to go into more depth on the approach to Risk Assessment, Assessment 
Scrutiny and Scoring. We would welcome your practical suggestions on how 
these aspects could be improved 

My suggestions for Risk Assessment are - to delete the section 

My suggestions for app scrutiny are — N/A 

My suggestions for scoring are (if you do not agree with scoring, please explain) — N/A 

Other (please specify) - The intention of the guidelines is to focus on validity and reliability. The questions should 
focus on what measure are taken to make sure that the app gives good and reliable data. Introducing risk into 
the equation will create confusion when considering current (and future) MEDDEV guidelines, Consumer 


Protection legislation and the current consultation on safety of non-embedded software. As stated before, it is 
the job of the European Commission through its defined processes to address this issue, not for this ad-hoc group. 


Question 11 - Appendices 

8.1. Health Evaluation and Standardisation Bodies existing in EU 
8.2. List of terms 

8.3. Assessment questionnaire 

8.4. Usability 

8.5. Definition of interoperability 


8.6. Case Studies 


Please make any comments on 8.1. Health Evaluation and Standardisation Bodies existing in EU. If you are aware 
of a body, you think should be mentioned - please reference it and explain why it should be included We do not 
understand the need and added value of this section. 


Please make any comments on 8.2. List of terms - This list of terms must be cross-referenced with legislation 
existing today that defines many of these categories. There are too many inconsistencies. 


Please make any comments 8.3. Assessment questionnaire. Bear in mind, we are still working on the questions, and 
especially how to make them less subjective and more objective. Please reference the specific question you are 
commenting on, and propose more objective wording if you feel able to do so (with justification for this) These 
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questions need to be drastically reduced. Many of these question have absolutely nothing to do with assessing 
whether an app produces credible, valid and reliable data. Furthermore, many questions contradict existing OS 
guidelines and will lead to confusion. This goes once again back to the scope issue that must be narrowed. 


Please make any comments on 8.4. Usability - We do not understand the need and added value of this section 


Please make any comments on 8.5. Definition of interoperability — This section is not needed unless there is a plan 
to introduce the details of specific standards for interoperability. We find this section randomly placed and of 
little added value. 


Please highlight any case studies (showing how similar guidelines have been adopted elsewhere and showcasing 
their impact). The Working Group is expecting to include case studies from Catalunia, Andalucia and the 
Netherlands, but more would be welcome. A maximum of one page of text is expected per case study — Any case 
studies should only include the section that address assess the reliability and validity of data. Other aspects of 
case studies are not relevant for this exercise and lead to confusion. 


Question 12 - Please provide any further comments you may have. Thanks for 
taking the survey and for your valued feed-back, which will help us further 
refine the guidelines 


As previously mentioned, it is important to remember that that DIGITALEUROPE, while only 1 member of the 
Working Group, represents 62 companies and 37 national trade associations. Our members employ 7.5 million 
people across the EU and generate €1.56 trillion. This impact must be taken into account and properly weighed 
when the responses to this questionnaire are analysed. 


For more information please contact: 
Damir Filipovic, DIGITALEUROPE’s Policy Director (Digital Enterprise & Consumer Policy) 
+32 2 609 53 25 or damir.filipovic@ digitaleurope.org 
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ABOUT DIGITALEUROPE 


DIGITALEUROPE represents the digital technology industry in Europe. Our members include some of the world's largest IT, 
telecoms and consumer electronics companies and national associations from every part of Europe. DIGITALEUROPE wants 
European businesses and citizens to benefit fully from digital technologies and for Europe to grow, attract and sustain the 
world's best digital technology companies. 


DIGITALEUROPE ensures industry participation in the development and implementation of EU policies. DIGITALEUROPE’s 
members include 62 corporate members and 37 national trade associations from across Europe. Our website provides 
further information on our recent news and activities: http://www.digitaleurope.org 


DIGITALEUROPE MEMBERSHIP 


Corporate Members 





Airbus, Amazon Web Services, AMD, Apple, BlackBerry, Bose, Brother, CA Technologies, Canon, Cisco, Dell, Dropbox, Epson, 
Ericsson, Fujitsu, Google, Hewlett Packard Enterprise, Hitachi, HP Inc., Huawei, IBM, Ingram Micro, Intel, iQor, JVC Kenwood 
Group, Konica Minolta, Kyocera, Lenovo, Lexmark, LG Electronics, Loewe, Microsoft, Mitsubishi Electric Europe, Motorola 
Solutions, NEC, Nokia, Nvidia Ltd., Océ, Oki, Oracle, Panasonic Europe, Philips, Pioneer, Qualcomm, Ricoh Europe PLC, 
Samsung, SAP, SAS, Schneider Electric IT Corporation, Sharp Electronics, Siemens, Sony, Swatch Group, Technicolor, Texas 
Instruments, Toshiba, TP Vision, VMware, Western Digital, Xerox, Zebra Technologies, ZTE Corporation. 


National Trade Associations 





Austria: |OO Germany: BITKOM, ZVEI Slovakia: ITAS 

Belarus: INFOPARK Greece: SEPE Slovenia: GZS 

Belgium: AGORIA Hungary: IVSZ Spain: AMETIC 

Bulgaria: BAIT Ireland: ICT IRELAND Sweden: Foreningen 

Cyprus: CITEA Italy: ANITEC Teknikféretagen i Sverige, 

Denmark: DI Digital, 1T-BRANCHEN Lithuania: INFOBALT IT&Telekomforetagen 

Estonia: ITL Netherlands: Nederland ICT, FIAR Switzerland: SWICO 

Finland: FFTI Poland: KIGEIT, PIIT, ZIPSEE Turkey: Digital Turkey Platform, ECID 
France: AFNUM, Force Numérique, Portugal: AGEFE Ukraine: IT UKRAINE 

Tech in France Romania: ANIS, APDETIC United Kingdom: techUK 
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